Walking Humanoids for Robotics Research

We present three humanoid robots aimed as platforms for research in robotics, and cognitive development in robotics systems. The 'priscilla' robot is a 180cm full scale humanoid, and the mid-size prototype is called 'elvis' and is about 70cm tall. The smallest size humanoid is the 'elvina' type, about 28 cm tall. Two instances of 'elvina' have been built to enable experiments with cooperating humanoids. The underlying ideas and conceptual principles, such as anthropomorphism, embodiment, and mechanisms for learning and adaptivity are introduced as well

Automated Speed and Lane Change Decision Making using Deep Reinforcement Learning

This paper introduces a method, based on deep reinforcement learning, for automatically generating a general purpose decision making function. A Deep Q-Network agent was trained in a simulated environment to handle speed and lane change decisions for a truck-trailer combination. In a highway driving case, it is shown that the method produced an agent that matched or surpassed the performance of a commonly used reference model. To demonstrate the generality of the method, the exact same algorithm was also tested by training it for an overtaking case on a road with oncoming traffic. Furthermore, a novel way of applying a convolutional neural network to high level input that represents interchangeable objects is also introduced

Combining Planning and Deep Reinforcement Learning in Tactical Decision Making for Autonomous Driving

Tactical decision making for autonomous driving is challenging due to the diversity of environments, the uncertainty in the sensor information, and the complex interaction with other road users. This paper introduces a general framework for tactical decision making, which combines the concepts of planning and learning, in the form of Monte Carlo tree search and deep reinforcement learning. The method is based on the AlphaGo Zero algorithm, which is extended to a domain with a continuous state space where self-play cannot be used. The framework is applied to two different highway driving cases in a simulated environment and it is shown to perform better than a commonly used baseline method. The strength of combining planning and learning is also illustrated by a comparison to using the Monte Carlo tree search or the neural network policy separately

Tactical Decision-Making in Autonomous Driving by Reinforcement Learning with Uncertainty Estimation

Reinforcement learning (RL) can be used to create a tactical decision-making agent for autonomous driving. However, previous approaches only output decisions and do not provide information about the agent's confidence in the recommended actions. This paper investigates how a Bayesian RL technique, based on an ensemble of neural networks with additional randomized prior functions (RPF), can be used to estimate the uncertainty of decisions in autonomous driving. A method for classifying whether or not an action should be considered safe is also introduced. The performance of the ensemble RPF method is evaluated by training an agent on a highway driving scenario. It is shown that the trained agent can estimate the uncertainty of its decisions and indicate an unacceptable level when the agent faces a situation that is far from the training distribution. Furthermore, within the training distribution, the ensemble RPF agent outperforms a standard Deep Q-Network agent. In this study, the estimated uncertainty is used to choose safe actions in unknown situations. However, the uncertainty information could also be used to identify situations that should be added to the training process

Effects of experience and electronic stability control on low friction collision avoidance in a truck driving simulator

Two experiments were carried out in a moving-base simulator, in which truck drivers of varying experience levels encountered a rear-end collision scenario on a low-friction road surface, with and without an electronic stability control (ESC) system. In the first experiment, the drivers experienced one instance of the rear-end scenario unexpectedly, and then several instances of a version of the scenario adapted for repeated collision avoidance. In the second experiment, the unexpected rear-end scenario concluded a stretch of driving otherwise unrelated to the study presented here. Across both experiments, novice drivers were found to collide more often than experienced drivers in the unexpected scenario. This result was found to be attributable mainly to longer steering reaction times of the novice drivers, possibly caused by lower expectancy for steering avoidance. The paradigm for repeated collision avoidance was able to reproduce the type of steering avoidance situation for which critical losses of control were observed in the unexpected scenario and, here, ESC was found to reliably reduce skidding and control loss. However, it remains unclear to what extent the results regarding ESC benefits in repeated avoidance are generalisable to unexpected situations. The approach of collecting data by appending one unexpected scenario to the end of an otherwise unrelated experiment was found useful, albeit with some caveats

A Review of Near-Collision Driver Behavior Models

Objective: This article provides a review of recent models of driver behavior in on-road collision situations. Background: In efforts to improve traffic safety, computer simulation of accident situations holds promise as a valuable tool, for both academia and industry. However, to ensure the validity of simulations, models are needed that accurately capture near-crash driver behavior, as observed in real traffic or driving experiments.<p> Method: Scientific articles were identified by a systematic approach, including extensive database searches. Criteria for inclusion were defined and applied, including the requirement that models should have been previously applied to simulate on-road collision avoidance behavior. Several selected models were implemented and tested in selected scenarios.<p> Results: The reviewed articles were grouped according to a rough taxonomy based on main emphasis, namely avoidance by braking, avoidance by steering, avoidance by a combination of braking and steering, effects of driver states and characteristics on avoidance, and simulation platforms.<p> Conclusion: A large number of near-collision driver behavior models have been proposed. Validation using human driving data has often been limited, but exceptions exist. The research field appears fragmented, but simulation-based comparison indicates that there may be more similarity between models than what is apparent from the model equations. Further comparison of models is recommended.<p> Application: This review provides traffic safety researchers with an overview of the field of driver models for collision situations. Specifically, researchers aiming to develop simulations of on-road collision accident situations can use this review to find suitable starting points for their work

Propeller optimization by interactive genetic algorithms and machine learning

Marine propeller design can be carried out with the aid of automated optimization, but experience shows that a such an approach has still been inferior to manual design in industrial scenarios. In this study, the automated propeller design optimization is evolved by integrating human–computer interaction as an intermediate step. An interactive optimization methodology, based on interactive genetic algorithms (IGAs), has been developed, where the blade designers systematically guide a genetic algorithm towards the objectives. The designers visualize and assess the shape of the blade cavitation and this evaluation is integrated in the optimization method. The IGA is further integrated with a support-vector machine model, in order to avoid user fatigue, IGA\u27s main disadvantage. The results of the present study show that the IGA optimization searches solutions in a more targeted manner and eventually finds more non-dominated feasible designs that also show a good cavitation behaviour in agreement with designer preference